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ABSTRACT 

A study evaluated an alternative seminar and major 
paper approach to the traditional doctoral program comprehensive 
examination at Pepperdine University (California). The pilot project 
was conducted over a period of 6 trimesters and involved 28 students 
who had already completed their course work. Students were also 
involved in the development of the evaluation criteria, which was 
then approved by the faculty. The approach followed was to require 
that students participated in a weekly seminar which focused on the 
development of a 30-50 page report addressing a real problem and 
including a plan or proposal to address that topic. The paper was 
then reviewed by three faculty members without knowledge of the 
student *s identity. The student later made an oral defense of his/her 
paper before the same faculty committee. Grades were composed of 
honors, pass, marginal pass, or fail. Post-program evaluation 
indicated that the students and faculty were overwhelmingly 
supportive of the seminar and paper approach. The approach was 
positively received by both students and faculty and appeared to have 
shortened the time for dissertation completion and was approved by 
the faculty to become an alternative to the classical comprehensive 
exams. (GLR) 
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STimnT OPTHPIOILII 

Ofer 10 yetri tfo, tke ficdt; it the Crtduite School of EdciCition lod Piycholof;, 
Pepperdite ttiTeriitj, were iiked to rote oa i proposed cktfife ii tke coipreheniife 
euiiiitioi proceif . Por nrioii retioBi the *chiO(e* ni lot tpprofed. Ihiriif the eniniof 
I jeiri, lefetil (tcilt; iciberii tlo&f litk itrotf icpport froi (rtdottei tid cqrrett 
itideiti it tke Iiitititioul liiifeiefit doctortl profrui developed it optioi to tke 
clMiicil cotprekeiiiTe eiui. Stsdeiti tid iiij ficvlt; referred to tke cltiiicil coipi ii 
t *re|ir(ititioi* of exm ih^idj ttket diritf coirie tork. Ai ft^ cotief leice of tke 
efforti to deii(i t ckufed tpproAck, t two pilot project wti iitrodiced it Septeiber 1190. 

imoioLon 

It tke fill of 1190 21 itidettir itfif coiiif oit of tke woodwork, ptrticiptted is tke 
iiitiftl pilot offeriif. Tke title of coiprekeiiive euiiiittioii vti ckttfed to 
CoipreketiiTe Seiiitr two ficiltf leiberi, eie it tick of two Pepperdiie ftcilitiei, 
lerved ti ftcilitttor/iettori. Tke itideiti ptrticipitiif at eick lite were iiTolTed it tke 
deTelopitit of tke iTtlittioi eriterit, wkick wti tbei tpproved bj tke ftciltj. Tke 
tpproftck followed wti to refiire tktt etck itedeit lelect a probiei/topic or iiiwe wkich 
woild refiire tke doTelopiett of i pUt or proposal to tddreii tktt topic. A ptper of $0 
ptfei or leii wti to be doTelopedi ijitbeiiiitf the doctortl coiriework. Tbii ptper wti 
reviewed by three ftciltj withoit ktowledge of the itideit'i ideititj. The itwdett Itter 
ttde tt ortl defeiie of hii/her ptper before the itie ftcilt; coiiittee. Grtdei were hotori, 
ptii, itrfittl ptii, or ftil. The pilot project wti cotdicted for lix triieiteri. 
COICUSIMS 

Stideiti ltd ftcwltf overwbeliiulj lupported the leiiitr tpprotch. Stideiti retcted 
itrotflp ii two treti: the tpprotch wti ftitlj iiperior to the cltiiictl tpprotch; tid, it 
wti cotiidertblf harder, bit worth it. Tbii tpprotck tlio tppetri to ktve iktrteted tke 
tiie for ditiertttioi coapletiot. Tke pilot profrti kti beet tpproved b; ftcilt; ttd will 
becoie tt tltertttiTe to the cltitictl coiprekeiiiTO eitii ii the text Pepperdiie Ctttlos* 
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1 Background 

The issue of whether or not comprehensive examinations 
accomplish their purpose has been debated for years 
throughout the academic community. Pepperdine ' s Graduate 
School of Education and Psychologr (GSEP) is no exception. 

In July of 1982, a proposal, prepared by two full-time 
faculty members, for an alternative approach to the 
comprehensive examinations was prepared and submitted for 
faculty consideration. This recommendation followed an 
evaluation of the doctoral program in institutional 
management ( referred to as the EDIM program) by an 
independent consultant , who recommended that alternative 
approaches be considered in view of the "unique nature of the 
program and its youthf ulness . " Quoting from the faculty 
proposal : 

"This process permits greater appreciation of course 
work in a less anxiety-provoking climate. Students will 
be expected to utilize skills and knowledge obtained 
from class work and apply them to a problem situation. 
This approach also enables close monitoring of student 
progress . Areas of weakness would be identified early 
in the weekly seminar and rectified by faculty 
assistance . " 

In the fall of 1982, at the Pepperdine campus on Vermont 
Avenue in Los Angeles , this proposal was submitted for 
faculty vote and was defeated. No further action was taken 
on this proposal until 1988 when expressions from students 
(through the EDIM Student Advisory Committee) and faculty 
(during Ed faculty meetings) revived the issue of identifying 
other approaches to the comprehensive examination process. 

For the ensuing academic year, discussions were held, both 
formally and informally, and two alternative approaches were 
considered. One involved a Charette-type environment and the 
other, patterned after the 1982 proposal, called for weekly 
seminar-type sessions with a paper and its oral defense being 
the final products. The focus of the seminar involved 
preparing a report that addressed a real problem. The 30 -50 
page report described a solution or a set of strategies to 
reach a solution to the problem. Students were expected to 
integrate and apply their doctoral coursework to the solution 
of the identified problem. The oral defense of the report 
offered faculty the opportunity to seek clarification of 
solution str Jitegies i address possible roadblocks to 
implementation, and ask "What if?" questions of the student. 
By faculty vote, both alternatives were considered viable. 



The second approach, for various reasons, was selected as an 
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initial pilot program and was designated to commence in the 
fall of 1990. 

2 Narrative Description of Initial (Fall 1990) Process 

Twenty eight students who had completed their course work 
expressed an interest in participating in this first pilot 
for the alternative approach to the comprehensive 
examinations. For the balance of this paper, this approach 
will be referred to as the Comprehensive Seminar. 



Of the initial 28 students, 15 enrolled at Pepperdine 
University Plaza in Culver City and 13 at the Oriinge County 
Center adjacent to the John Wayne Airport. Two students 
subsequently dropped from the Orange County group. 

A copy of the initial handout for the fall o2 1990 appears as 
Appendix A to this paper . Although modifications were made 
during the two year pilot term, the major change in conduct 
of the pilot was that in the fall of 1990 the students were 
asked to develop the criteria against which their papers and 
oral defense would be evaluated. This process took the first 
three weeks of the trimester, but did result in the criteria 
forms. Slight modifications were made to these two forms 
(written and oral materials) during the first three 
trimesters of the pilot Comprehensive Seminar program. 

3 Narrative Description of Spring 1992 Process 

During the final trimester of the pilot Comprehensive S3minar 
program, 10 students were enrolled at the Orange County 
Center. A copy of the initial handout describing the course, 
its objectives, schedule and additional notes appears as 
Appendix B. 

Over the six trimesters of conducting the Comprehensive 
Seminar pilot, the two seminar leaders developed hints and 
suggestions for the students • 

Briefly, the following describes the steps for the 
Comprehensive Seminar pilot during .he spring trimester of 
1992: 

1* During the first few weeks, students working with 
the seminar leader and other students, define a 
problem, issue, or dilemma which will serve as the 
topic for the Comprehensive Seminar. NOTE: for some 
students this is quite difficult. During the spring 
trimester, one student did not successfully identify 
his/her topic until the sixth week of the seminar. 
The topic needs to have sufficient breadth to permit 

consideration of all coursework but Also must be 
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sufficiently narrow that a solution can be reached* 

Once the topic has been identified , the students 
commence preparation of their paper which is a plan 
or a proposal (not chapters 1, 2, and 3 of a 
dissertation proposal ) . They are instructed to 
assume the role of a decision-maker in the 
organization and to prepare this plan to be 
presented to a group of persons (pe'rhaps board of 
trustees or regents). This plan is to synthesize 
the course work taken during the EDIM program. This 
is the second difficult time for the students. Many 
of them still think of their courses in neat boxes 
and have trouble tying the content together into a 
cohesive document . 

There are key milestones for student submissions for 
review by the seminar leader. Students adhere to 
these timelines differently . For example , some 
students will not submit their paper until they are 
almost completed, thereby providing little time for 
the seminar leader to make constructive criticisms . 

Two weeks before the oral defense of the paper, 
students submit their paper for faculty review. 
This review is conducted without an identification 
of the student ^s name. Clearly, some topics 
selected by students make it easy to identify whose 
paper is being reviewed. Faculty have been 
identified for the three person review committees by 
the staff Program Coordinator with every effort 
being made to assure that at least one faculty 
member who has taught the student is on the 
committee. During this two week period, students 
ai^e subjected to mock orals before their seminar 
leader and fellow students. 

Three days before the oral defense of the papers 
(now scheduled for the next to last Thursday and/or 
Friday of the trimester) students are aware of who 
is on their committee and faculty are aware, 
formally, of whose paper has been reviewed. 

Faculty evaluations are given to the Program 
Coordinator and placed in a sealed envelope. At the 
time of the oral defense, the envelope is given to 
the chair of the committee. The envelope remains 
sealed until the orals are completed and the oral 
evaluation submitted to the chair by each committee 
member . At that time , the committee chair examines 
both the Peeper evaluations and the oral evaluations. 



If there is "harmony" on the part of the committee 
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members, then a grade of honors, pass, marginal 
pass , or fail is reported following all oral 
defenses. A marginal evaluation suggests a need for 
some modification to the submitted paper and 
generally is followed by giving an "IP" (In 
Progress) grade rather than a "P" (Pass) grade. 
Since the start of the Comprehensive Seminar pilot 
program there have been four failures and five 
honors, with 60 students being involved over the two 
year period. 

As the trimester progresses, faculty and students receive 
additional documentation : 

1. For the faculty, the sequence of events (just a 
reminder ) 

2. For the faculty, conduct of the orals (some 
observations as faculty prepare for the orals) 

3. Some instructions for the Chair 

4. Requirements for submission of materials by the 
students . 



4 Summary of Evaluations: 

Following completion of their orals, each student and faculty 
member is asked to complete evaluation forms. Faculty 
provide an overall evaluation while the students evaluate the 
Comprehensive Seminar pilot, as well as the orals On each 
for,^ the following four items appear: 

1. I liked 

2. Next time 

3. My overall rating of the process is 

4 . I will suggest to future candidates th/it 

Following are selected observations from the student and 
faculty comments. In reviewing these detailed comments, it is 
important to note that suggestions made by both students and 
faculty, where possible, were incorporated into the process. 

4 . 1 • Faculty 

1. I liked: 

Faculty interaction , 

Meeting students , talking with colleagues . 

A chance to probe the thinking of our students . 



2 • Next time : 
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Find a method to reduce anxiety. 

Have students start to write papers sooner. 

Place more emphasis on quality of written report. 

Consider a simulation process. 

Relate better to Institutional Management, 

Give us clearer criteria on evaluating papers and orals, 

Let^s work on rating sheets. 



3 , My overall rating of the process is : 



Most comments are great, excellent, good, very good, 

1 will suggest to future candidates that : 

They select a broad enough problem to incorporate course 
work , 

They practice synthesis throughout course work. 
They improve their writing skills. 

They begin thinking about the synthesis process before 
the Comprehensive Seminar, 



4,2* Students, Pilot 
1. I liked 



The feed back from seminar leaders. 
The interaction with the group. 
The entire process. 

Opportunity to review and synthesize course materials. 
The mock orals process, 

2 . Next time 



Start projects earlier . 

Have faculty members visit seminar and discuss critical 
issues . 

More direction/earlier critiques. 



3 . My overall rating of the process is 



Reactions were positive, with comments such as "time 
consuming, but relevant," Now I appreciate the process," 
and "outstanding • " » x 



4 . X will suggest to future candidates that 

Plan scenarios before Comprehensive Seminar 
starts • 

They opt for the process. 

They participate in this model. 



4*3* Students, Orals q 
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I liked 



Panel's comments were very good. 

Positive tone set by panel. 

Open, relaxed atmosphere of faculty. 

The types of questions asked and the way follow-up 

questions were asked for clarification. 

Next time 



Have at least two committee members know the student. 
Continue the "mock orals" 

Perhaps allow candidate to select one committee member. 
Place more emphasis on preparation for the orals. 
Some of us felt we were being raked over the coals at 
the orals. 

Please, God, don't let there be a next time. 

My overall rating of the process is 

Wow! A wonderful learning experience. 

Ratings were all high: good, very good, excellent, A+ . 
X will suggest to future candidates that ..... 
Be relaxed and prepared. 

Begin early, keep on schedule, and develop an overall 
plan. 

Keep an accurate system of class notes. 

Observations by the Seminar Leaders 

Since the Comprehensive Seminar appears to be positively 
accepted by both faculty and students, it should be 
adopted as an option to the classical comprehensive 
examinations. It was in the late Spring of 1992 and 
becomes a catalog entry during the 1993-94 academic year. 

Although an inter-rater reliability assessment was done 
during one trimester, it appears that faculty are rating 
students from differing perspectives. Therefore, a more 
specific set of evaluation criteria should be developed, 
using the forms currently in use as the starting point. n 
What, for example, differentiates an honors paper from a 
pass paper? 

More faculty need to be involved throughout the process, 
both in conducting the seminars and in the evaluation of 
the papers and oral defenses. 

Although weekly seminar sessions were held during the 
pilot, it may be appropriate to consider sessions which 
are longer but held less frequently. This is being 
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tested during the current, fall 1992, trimester • 

5* Consideration needs to be given to how we should be 

dealing with the marginal student who cannot write well 
or develop a logical approach. How, for example, should 
we be handling the student rated as "marginal'* by the 
faculty committee c 

6, Both seminar leaders consider the load generated by this 
approach considerably greater than an equivalent two unit 
course, particularly when there are more then seven or 
eight students in the seminar. An adequate compensation 
schedule must be developed* 

7, Time is a critical element. The expectation of reaching 
a potential solution to a real-world problem and 
preparing a report may take more than the 12 weeks. Many 
students experience difficulty in synthesizing and 
applying their course work over this short time period, 

8, Faculty are becoming more aware of the content of each 
other's courses by reading the papers and participating 
in the orals. It is unusual for many faculty members to 
sit together as a team to "discuss" the applications to 
real-world problems of doctoral coursework with a 
student . 

9, The need for individual professors to alter their 
teaching styles and model application of theory to 
practice is becoming very apparent. Questions now arise 
in post-oral discussions regarding the "how" and "what" 
of the doctoral coursework, 

10 • This approach helps students in the preparation of their 
dissertations. Some students continue with the same 
topic and complete a research study related to some 
identified aspect of the comprehensive report. Others 
become more skilled during the oral defense of their 
solution, thereby assisting them greatly with the 
subsequent oral defenses at the dissertation stage. 
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